Hard-to-Test Code

The book has now been published and the content of this chapter has likely changed substanstially.
Please see page 209 of xUnit Test Patterns for the latest information.

Code is difficult to test.

Automated testing is a powerful tool that helps us develop software quickly even after we have a large code base to maintain but it only provides these benefits if most of our code has Fully Automated Test (see Goals of Test Automation on page X). The effort of writing these tests is in addition to the effort of writing the product code they verify so we would like to make it easy to write the automated tests(We would also like to recoup this cost by reducing effort somewhere else. The best way to achieve this is to avoid Frequent Debugging (page X) by writing the tests first and achieving Defect Localization (see Goals of Test Automation).).

Hard-to-Test Code is one factor that makes it hard to write complete, correct automated tests in a cost-efficient manner.

Symptoms

Some kinds of code are difficult to test. GUI components, multi-threaded code and test code come immediately to mind as Hard-to-Test Code. It may be difficult to get at the code to be tested because it is not visible to a test. It may be hard to get a test to compile because the code is too highly coupled to other classes or it may be hard to create an instance of the object because the constructors don't exist, are private, or take too many other objects as parameters.

Impact

Any time we have Hard-to-Test Code, we have code whose quality we cannot easily verify in an automated way. While manual quality assessment is often possible, it doesn't scale very well because the effort to do it after each code change usually means it doesn't get done. Nor is it very repeatable without a large test documentation cost.

Solution Patterns

A better solution is to make the code more amenable to testing. This is a big enough topic that it warrants a whole chapter of its own but I'll cover a few of the highlights here.

Causes

There are a number of reasons for Hard-to-Test Code but the most common causes are:

Cause: Highly Coupled Code

Also known as: Hard-Coded Dependency

Symptoms

A class cannot be tested without also testing several other classes.

Impact

Code that is highly coupled to other code is very difficult to unit test because it won't execute in isolation.

Root Cause

Highly Coupled Code can be caused by many factors including poor design, lack of object-oriented design experience or lack of a reward structure that encourges decoupling.

Possible Solution

The key to testing overly coupled code is to break the coupling. This happens naturally when doing test-driven development.

A technique that we often use to decouple code for the purpose of testing is the Test Double (page X) or more specifically, Test Stubs (page X) or Mock Objects (page X). This topic is covered in much more detail in the Using Test Doubles narrative chapter.

It is more challenging when retrofitting tests onto existing code, especially when we are dealing with a legacy code base. This is a big enough topic that Michael Feathers wrote a whole book on techniques for doing this. It is called "Working Effectively with Legacy Code" [WEwLC].

Cause: Asynchronous Code

Symptoms

A class cannot be tested via direct method calls. The test must start up an executable (such as a thread, process or application) and wait until it has finished starting up before interacting with it.

Impact

Code that has an asynchronous interface is hard to test because the tests must co-ordinate their execution with that of the system under test (SUT). This can add a lot of complexity to the tests and will also make them take much, much longer to run. The latter is a major issue for unit tests which must run very quickly to ensure that developers will run them frequently.

Root Cause

This is specific form of coupling where the code that implements the algorithm we wish to test is highly coupled to the active object in which it normally executes.

Possible Solution

The key to testing asynchronous code is to separate the logic from the asynchronous access mechanism. The design-for-testability pattern Humble Object (page X) (including Humble Dialog and Humble Executable) is a good example of a way to restructure otherwise asynchronous code so it can be tested in a synchronous manner.

Cause: Untestable Test Code

Symptoms

The body of a Test Method (page X) is obscure enough (Obscure Test (page X)) or contains enough Conditional Test Logic (page X) to cause us to wonder whether the test is correct.

Impact

Any Conditional Test Logic within a Test Method has a higher probability of resulting in Buggy Tests (page X) and will likely result in High Test Maintenance Cost (page X). Too much code in the test method body can make the test hard to understand and hard to get right.

Root Cause

The code within the body of the Test Method is inherently hard to test using a Self-Checking Test (see Goals of Test Automation). We'd have to replace the SUT with a Test Double that injects the error we are testing for and then run the test method inside another Expected Exception Test (see Test Method) method; much too much hassle to bother with in all but the most unusual circumstances.

Possible Solution

We can remove the need to test the body of a Test Method by making it extremely simple and removing any Conditional Test Logic from it into Test Utility Methods (page X) for which we can easily write Self-Checking Tests.

Page generated at Wed Feb 09 16:39:51 +1100 2011